The National Science Foundation (NSF) has awarded MBARI $5 million for Ocean Vision AI, a new program that leverages artificial intelligence and machine learning to accelerate processing of—and access to—ocean video and imagery to enable effective marine stewardship.

Ocean Vision AI combines the expertise of MBARI, the Central and Northern California Ocean Observing System (CeNCOOS), Climate Change AI, CVision AI, Ocean Discovery League, and the University of California, Santa Cruz, to create a machine-learning platform that will aid the processing of underwater visual data with a globally integrated network of services, tools, and diverse community of users. Ocean Vision AI’s scope goes beyond data aggregation and processing pipelines. This project will also develop a video game to cultivate a community of ocean enthusiasts that can help improve machine-learning models.

“MBARI and our collaborators are excited to leverage the power of artificial intelligence, community science, and gaming to accelerate the analysis of ocean data. Together, we’re developing tools that are urgently needed to help us better understand and protect our blue planet,” said Kakani Katija, a principal engineer at MBARI and the lead principal investigator for Ocean Vision AI.

Multidisciplinary collaborators and supporting team members that include data scientists, oceanographers, game developers, and human-computer interaction experts will streamline access and analysis of ocean visual data to promote effective marine stewardship.

“Only a fraction of the hundreds of thousands of hours of ocean video and imagery captured has been viewed and analyzed in its entirety and even less shared with the global scientific community,” said Katy Croff Bell, founder and president of the Ocean Discovery League and a co-principal investigator for Ocean Vision AI. “In a single dive, thousands of species could have been recorded in new locations, at new depths, and new environmental conditions. A wealth of untapped biodiversity information is trapped in these existing recordings, with new data acquired daily.”

Analyzing visual data—particularly data with complex scenes and organisms that require expert classifications—is a resource-intensive process that is not scalable in its current form. As more visual data are collected, the ocean community faces a data analysis backlog that artificial intelligence can help solve.

The new Ocean Vision AI will:

• Provide a central hub for groups conducting research using underwater imaging, artificial intelligence, and open data;
• Create data pipelines from existing image and video data repositories;
• Provide project tools for coordination;
• Leverage public participation and engagement via game development; and
• Generate data products shared with researchers and other open-data repositories.

Researchers and scientists studying ways to conserve ocean resources, and anyone interested in what happens below the ocean surface, will be able to access the resulting verified annotations and trained algorithms via open-source platforms like FathomNet. In the future, this new technology has the potential to usher in real-time processing of ocean imagery, scaling up our observation capabilities using current and future robotic technologies.

Ocean Vision AI also seeks to build a global community, from enthusiasts to experts, around underwater visual data akin to an iNaturalist or eBird for the ocean.

Advancements in machine learning and artificial intelligence can help process the deluge of data researchers are collecting about the ocean, but still require human intervention to train, evaluate, verify, and improve the performance of these algorithms. Ocean Vision AI will develop game-based human annotation pipelines to engage a broader audience. A video game will teach casual gamers about the ocean while improving machine-learning models and expanding annotated datasets.

“Ocean Vision AI will facilitate a dynamic ebb and flow of information,” said Katija. “Not only are people learning about the ocean, but they’re also contributing directly to our understanding of the ocean and helping us improve ocean AI.”

A long-term goal of Ocean Vision AI is not only to accelerate ocean discoveries in video and imagery but also to provide these findings and the tools used to support them to a much wider section of the ocean community.

“Ocean Vision AI will unleash a raft of information that will benefit scientists and resource managers. Part of our challenge is to work with users of resulting information so that it’s most useful to them and it comes in ways that users understand what they’re getting,” said Henry Ruhl, CeNCOOS director and a co-principal investigator for Ocean Vision AI.

“The ocean plays a vital role in the health of our planet, yet we have only observed a tiny fraction of it,” said Katija. “Technology innovations like Ocean Vision AI are critical to accelerating the exploration of the ocean and addressing the long-standing challenge of enabling large-scale biological observations in the marine environment so we can sustainably manage and care for this incredible, shared resource.”

Funding for Ocean Vision AI is part of the second phase of NSF’s Convergence Accelerator Track E: Networked Blue Economy. More information is included in NSF’s Convergence Accelerator 2022 Portfolio Guide.